1. Spectral Embedding-Based Active Contour (SEAC) for Lesion Segmentation
Spectral Embedding (SE)
Spectral embedding (SE), a graph-based manifold learning method, has been shown to be useful in high dimensional data classification. SE is used to cluster image pixels based on their similarity to provide an alternative data representation to a multidimensional image. The SE image representation has the ability to visualize the information of greater than two dimensions at each pixel location based on its similarity to other pixels in the image, and the output of SE is an orthonormal basis of eigenvectors such that the information in each eigenvector is independent, yet captures the fundamental patterns in the image. The output of SE can be used to segment, or delineate, a region of interest (ROI) in the image from non-ROI areas in the image. It can also be used to register, or align, images based on a set of features calculated at each pixel location in the image.
Active Contour (AC)
Active contour (AC) methods are commonly used for medical image segmentation. While AC methods are often successful in object segmentation, their success hinges upon the existence of strong signal intensity gradients at the interface between the region of interest and its surrounding structures. This is often a disadvantage of the AC model and poses a particularly difficult problem for grayscale images such as those found in radiologic images. Because radiologic images are composed of subtle changes in grayscale signal intensity, a strong gradient can be difficult to define, particularly when the region of interest has diffuse boundaries. In addition, in time-series data such as dynamic contrast enhancement magnetic resonance imaging (DCE-MRI), the question of which time point to use for lesion segmentation poses a difficult problem.
In general, AC models can either be boundary-based or region-based (Chan, T. and Vese, L., Image Processing, IEEE Transactions on 10, pp. 266-277, 2001). One of the key components of a boundary-based AC model is the energy term, which requires strong gradients at the boundary between the region of interest (ROI) and the background. Since the boundary of the ROI does not always have a strong gradient, alternative methods to derive the energy term have been explored. Gradient vector flow was proposed to incorporate the gradient field of the image into the energy term so that when the magnitude of gradient field was small, the partial derivative dominates the term, whereas the gradient field magnitude dominates the energy term when its magnitude is large.
Manifold learning based schemes like spectral embedding (SE) allow for parametrically representing high dimensional data in an alternative embedding space. However, while several researchers have explored SE for data classification (Eyal, E. et al., J Magn Reson Imaging 30, 989-998, 2009; Jamieson, A. et al., Med Phys, 37, 339-351, 2010; Lee, G. et al., IEEE/ACM Transactions on Computational Biology and Bioinformatics 5, 368-384, 2008), few studies have addressed alternative data representations for facilitating segmentation of time dependent MRI such as DCE-MRI. Eyal et al. (J Magn Reson Imaging 30, 989-998, 2009) used the principal eigenvectors derived from principal component analysis (PCA) to determine a parametric representation of breast DCE-MRI data. Since the feature matrix in PCA is a covariance matrix, transforming data in the PCA space and using the eigenvectors associated with the largest eigenvalues rotates the data along axes of maximum variance. Thus, if one were to use the PCA eigenvectors as the gradient functional, the stopping criterion would be based on a gradient of deviation from the mean, which would not provide a strong gradient. In contrast, SE uses the eigenvectors corresponding to the minimum eigenvalues based on the eigenvalue decomposition of a weighted similarity matrix, providing an intuitive strong gradient for the AC. In addition, the SE transformed eigenvectors not only preserve both local and global similarities, but also are orthonormal to another so that each eigenvector contains independent information.
Lesion Segmentation on Breast Dynamic Contrast Enhanced Magnetic Resonance Imaging (DCE-MRI)
Breast lesion segmentation is an important pre-processing step in a computer aided diagnosis framework for breast dynamic contrast enhanced (DCE) magnetic resonance imaging (MRI). Several studies have shown that quantitative morphological features extracted from breast lesions are helpful for distinguishing between benign and malignant breast lesions. (Ikeda D. et al., Journal of Magnetic Resonance Imaging, 13:889-895, 2001; Schnall, M. et al., Radiology, 238(1):42-53, 2006.). Typically, a radiologist's expert delineation of the lesion boundary is considered the gold standard for lesion segmentation. However, manual segmentation is notoriously susceptible to inter-rater variability in breast MRI interpretation (Ikeda D. et al., Journal of Magnetic Resonance Imaging, 13:889-895, 2001; Kinkel, K. et al., Am. J. Roentgenol., 175(1):35-43, 2000) and is extremely time consuming.
Because accurate lesion segmentation is time consuming, yet necessary for quantitative lesion analysis, many groups have explored various automated segmentation methods for breast dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) (Chen, W. et al., Medical Physics, 33(8):2878-2887, 2006; Shi et al., Medical Physics, 36(11):5052-5063, 2009; Botond, K. et al., Academic Radiology, 11:1344-1354, 2004; Twellmann, T. et al., Eng. Appl. Artif. Intell., 21:129-140, 2008; Wu, Q., et al., SPIE, volume 6144, page 61444M. 2006.). Automated lesion segmentation methods for breast dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) have been explored mostly in the context of pixel-wise clustering of the data. Szabo et al. (Szabo, B. et al, Academic Radiology, 11:1344-1354, 2004) used a pixel-wise classifier that used dynamic contrast signal intensities in conjunction with an artificial neural network to identify lesion areas of interest. Other methods that have also used pixel-wise classifiers for segmentation include Twellmann et al. (Twellmann, T. et al., Eng. Appl. Artif. Intell., 21:129-140, 2008), who used the dynamic contrast signal intensities in conjunction with a support vector machine (SVM) classifier and Chen et al (Chen, W. et al., Medical Physics, 33(8):2878-2887, 2006.), who used a fuzzy c-means (FCM) clustering scheme. Additionally, Wu et al. (SPIE, volume 6144, page 61444M, 2006) clustered the time-series data of breast dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) using Markov random fields. Although these pixel-wise methods are reasonably effective, most require post-processing morphological operations, such as hole-filling and dilation, in order to provide a closed contour for the lesion of interest.
An alternative to pixel-wise methods are shape-based deformable models, most popular of which is the active contour (AC) model (Caselles, V. et al., International Journal of Computer Vision, 22:61-79, 1997. 10.1023/A:1007979827043; Chan, T. and Vese, L., Image Processing, IEEE Transactions on, 10(2):266-277, February 2001; Tony, F. et al., Journal of Visual Communication and Image Representation, 11(2):130-141, 2000; Kass, M. et al., International Journal of Computer Vision, 1(4):321-331, 1988; Rousson, M and Deriche, R., In Motion and Video Computing, 2002. Proceedings. Workshop on, pages 56-61, December 2002; Sapiro, G. Color snakes. CVIU, 68(2):247-253, 1997; Xu, J. et al., Medical Image Analysis, In Press: Corrected Proof, 2011).
The theory of the active contour (AC) model, introduced by Kass et al. (International Journal of Computer Vision, 1(4):321-331, 1988), is that (1) the segmentation of any object of interest (OOI) in an image, whose edges can be described by a closed curve, is equivalent to the location of edges, or sharp intensity gradients; and (2) this segmentation can be generated by iteratively deforming a curve towards the edges of the object of interest (OOI). Traditional AC models have been typically classified as: (1) boundary-based (Caselles, V. et al., International Journal of Computer Vision, 22:61-79, 1997; Kass, M. et al., International Journal of Computer Vision, 1(4):321-331, 1988), such as the AC described by Kass et al. International Journal of Computer Vision, 1(4):321-331, 1988), or (2) region-based methods (Chan et al., Image Processing, IEEE Transactions on, 10(2):266-277, February 2001.). However, to use an AC model, an image representation that is conducive to the stopping criteria of the curve evolution is necessary. For example, boundary-based methods require strong gradients located at the boundary of the object of interest (OOI) to provide an effective stopping criterion for the evolving AC model. For radiologic imaging applications involving MRI or computed tomography (CT) data, boundary-based methods may not be effective since image acquisition artifacts, such as partial volume effects (Bradley, W. and Glenn, B., American Journal of Neuroradiology, 8(6):1057-62, 1987; Glover, G. and Pelc, N., Nonlinear partial volume artifacts in x-ray computed tomography, 7(3):238-248, 1980) and a low signal to noise ratio, may result in fuzziness of the object boundary, preventing boundary-based AC models from having appropriate stopping criteria. Region-based methods rely on the image statistics of foreground and background regions in the image. Hence, a grayscale radiologic image may not provide a large enough difference between foreground and background image statistics to provide an effective stopping criterion for a region-based AC either. Consequently, there is a need to find alternative image representations that can provide (1) strong gradients at the margin of the object of interest; and (2) larger separation between intensity distributions and region statistics for the foreground and the background which would cause the AC to stop evolving at the border of the OOI.
Alternative image representations have been explored for noise filtering (Saha, P. and Udupa, J., Medical Imaging, IEEE Transactions on, 20(11):1140-1155, November 2001), image registration (Nyul, L. et al., IEEE Trans Med Imaging, 22(2):228-237, 2003), and fuzzy connectedness based image segmentation (Punam, K. et al., Computer Vision and Image Understanding, 77(2):145-174, 2000). Nyul et al. (IEEE Trans Med Imaging, 22(2):228-237, 2003) employed ball-scale for multi-protocol image registration, where ball-scale (IEEE Trans Med Imaging, 22(2):228-237, February 2003) is a locally adaptive scale definition such that every image pixel location is parametrized by the radius of the largest ball that satisfies some pre-defined local homogeneity criterion. Saha (Computer Vision and Image Understanding, 99(3):384-413, 2005) defined tensor scale (t-scale) at every spatial location as the largest ellipse that satisfies some pre-defined homogeneity criterion. The t-scale based representation has been employed in the context of image segmentation and image filtering (Saha, P. et al., Tensor scale-based image registration. volume 5032, pages 314-324. SPIE, 2003; Saha, P. et al., Computer Vision and Image Understanding, 99(3):384-413, 2005)
The generalized scale (g-scale) (Madabhushi, A. et al., Computer Vision and Image Understanding, 101(2):100-121, 2006) has also been applied similarly to bias field correction (Madabhushi, A. et al., Computer Vision and Image Understanding, 101(2):100-121, 2006), noise filtering (40), and intensity standardization (Madabhushi, A. et al., Medical Physics, 33(9):3426-3434, 2006). In each of these methods, transforming the data into another image space allowed for an improvement in the corresponding image processing task.
Nonlinear dimensionality reduction (NLDR) is a computational method that transforms data from a high-dimensional space to a more manageable, low-dimensional space and can be particularly powerful in data visualization (Hamarneh, G. et al., Medical Imaging, IEEE Transactions on, 30(7):1314-1327, 2011) and classification (Eyal, E. et al., J Magn Reson Imaging, 30(5):989-998, 2009; Jamieson, A. et al., Med Phys, 37(1):339-351, 2010; Lee, G. et al., IEEE/ACM Transactions on Computational Biology and Bioinformatics, 5:368-384, 2008). Spectral embedding (SE), a type of NLDR, uses the eigenvectors corresponding to the minimum eigenvalues derived from the eigenvalue decomposition of a weighted affinity matrix (Shi, J. and Malik, J., IEEE PAMI, 22(8):888-905, 2000), where the affinity matrix represents the pairwise dissimilarity between all the objects to be classified, obtained via a Gaussian, exponential, or polynomial kernel in the original feature space. SE also allows for parametrically representing high dimensional data in a reduced dimensional space, and several researchers have employed SE in the context of image partitioning (Meila, M. and Shi, J., A random walks view of spectral segmentation. 2001; Shi, J. and Malik, J., IEEE PAMI, 22(8):888-905, 2000), clustering (Weiss, Y., Computer Vision, IEEE International Conference on, 2:975, 1999), and segmentation. Application of SE to time series or longitudinal data may allow for better capture and representation of both region and boundary-based statistics. In addition, improved region and boundary statistics in turn may allow for construction of improved hybrid active contour schemes. However, no attempts have been made to explore the utility of nonlinear dimensionality reduction (NLDR) schemes to seek improved image representations that would be amenable for use in conjunction with an AC based segmentation scheme.
The traditional active contour (AC) operates on the scalar grayscale image intensities. However, time series data, such as dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI), contains multiple time points over which the image of the lesion of interest is captured. Typically, if a traditional AC is used, only a single time point (usually the time point at which the lesion maximally enhances) is used for segmentation. However, implementations of the AC model have been developed for multi-dimensional images (Tony, F. et al., Journal of Visual Communication and Image Representation, 11(2):130-141, 2000; Rorden, C. and Brett, M., Behav Neurol, 12(4):191-200, 2000; Sapiro, G., CVIU, 68(2):247-253, 1997; Xu, J. et al., Medical Image Analysis, In Press: Corrected Proof, 2011). Chan et al. (Journal of Visual Communication and Image Representation, 11(2):130-141, 2000) demonstrated an extension of the original scalar image based AC model applied to vector-valued images. Rousson and Deriche (In Motion and Video Computing, 2002. Proceedings. Workshop on, pages 56-61, December 2002) also demonstrated a vector-valued active contour. In a recent application to a histological medical imaging problem, Xu et al. (Medical Image Analysis, In Press: Corrected Proof, 2011) developed a tensor gradient based AC for use with histopathological images by computing the gradient from vectorial images, and hence representing the image gradient as tensors. Xu et al. (Medical Image Analysis, In Press: Corrected Proof, 2011) showed that the tensor gradient more completely captured the gradient information in a multi-channel image than using a single channel of the image, yielding a more discriminating AC scheme.
Spectral embedding (SE) aims to partition the data points in a way that maximizes intra-cluster similarity while simultaneously minimizing inter-cluster similarity (Shi, J. and Malik, J., IEEE PAMI, 22(8):888-905, 2000), and the eigenvectors are oriented along the directions of fundamental patterns in the data. In the context of dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI), these fundamental patterns are related to the time-intensity curves at each pixel in the image, and the time-intensity curves from lesion and non-lesion areas tend to have different characteristics as previously shown in multiple different studies (Chen, W. et al., Medical Physics, 33(8):2878-2887, 2006; Botond, K. et al., Academic Radiology, 11:1344-1354, 2004).
Therefore, by applying SE across all pixels in an image, the described invention aims to characterize pixels according to their time-intensity curves, an approach that has not been taken before with respect to dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) data. In addition, pixel similarity is reflected by the eigenvectors at each pixel such that pixels with similar eigenvector values have similar time-intensity curves. Since this is performed in a pixel-wise fashion, the image scene composed of the eigenvector values reflect region similarities and global differences in the images (see FIG. 1).
This alternative image scene information driven by the intensity profiles provides greater knowledge for approximating the region statistics of the image via SE's ability to preserve global data information (Shi, J. and Malik, J., IEEE PAMI, 22(8):888-905, 2000). Similarly, the alternative image scene information resulting from SE provides better boundary information by preserving the local image information as well as constraining the data in such a way that the distances between pixel clusters with different time-intensity curve profiles will be maximized. Recently, Eyal et al. (J Magn Reson Imaging, 30(5):989-998, November 2009) used the principal eigenvectors derived from principal component analysis (PCA) to determine a parametric representation of breast dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) data for lesion classification.
2. Spectral Embedding-Based Registration (SERg) for Time Series Imaging
Alternative data representations for improved registration accuracy and efficiency have been explored with the idea that two images in the transformed space may be more similar than the corresponding images in the raw intensity space. Nyul et al. (EEE Trans. Med. Imag. 22(2), 228-237, 2003) used the ball-scale concept, where a ball of a certain radius defined a region of homogeneity in an image, to drive registration. Similarly, Saha (Tensor scale-based image registration. Volume 5032., SPIE, 314-324, 2003) demonstrated the effectiveness of tensor scale (t-scale), whereby an ellipsoid defines the region of homogeneity, to improve image registration.
Although alternative data representations have been developed previously primarily for motion correction, time-series data involving dynamic contrast enhancement (DCE) introduces another difference between target and template images, i.e., changes in contrast. Since the post-contrast images are registered typically to pre-contrast images, comparing images whose dynamic intensity ranges change from time point to time point on the basis of signal intensity is not an appropriate method for dynamic contrast enhanced (DCE) data. This is illustrated in FIG. 9, which shows the subtraction images of a dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) of the breast with and without intensity based registration at three different time points.